Update to hash-to-curve draft 16, with some API adjustments #90

andrewwhitehead · 2022-07-05T18:45:55Z

Adds a Message trait for use by expand_message rather than requiring the message to be a single [u8] (more flexibility for larger message values and no-alloc situations)
Allows expand_message to be used with security levels other than k=128
Removes lifetime parameters and the separate InitExpandMessage trait
Moves unit tests to tests/ for compilation performance

tests/hash_to_curve_g1.rs

str4d · 2023-02-27T15:19:30Z

Hey! This PR has been close to the top of our to-review list for months now, but just as it keeps getting close to "hey, we can start working on this now!" some other emergency pushes it back down. Part of the problem is our desire to review the delta from draft 12 to draft 16 and confirm there are no backwards-incompatible changes, but also this PR is very big and requires significant uninterrupted time to review (even though most of it appears to only be test moves, it does add to the review complexity).

I strongly want to get this in, and will continue to try and do so, but we do not have time to review it before the next crate release.

str4d · 2023-02-27T16:38:11Z

On the above point, I think it would be significantly easier to review this PR if it was split into two separate PRs: a move-only PR that moves the tests to tests/, and then this PR changing the API.

The `pairing` feature from the `fvm_shared` crate isn't used. It causes problems, as it forces the `subtle` dependency to v2.4.1, although the rest is happy to have v2.5.0. Here is a detailed dependency graph and issue outline: `fvm_shared` depends on `bls-signatures`. In `bls-signatures` we depend on an old version (v0.11) of `hkdf`. That version depends on `hmac` v0.11, which depends on `crypto-mac` v0.11. `crypto-mac` v0.11.0 depends on `subtle` v2. That is fine, it would automatically select v2.5.0. The problem is that `crypto-mac` v0.11.1 pins `subtle` to exactly v2.4, therefore v2.5.0 won't be selected. The obvious thing is to upgrade in`bls-signatures` the version of `hkdf` to the latest v0.12. That would make it possible to use `subtle` v2.5.0. The problem is that such an upgrade is not easily possible. `hkdf` v0.12 depends on a newer version v0.10 of the `sha2` crate. Updating that breaks the `bls12_381` crate. The reason is the current version v0.8.0 of `bls12_381` depends on an old version v0.9 of the `digest` crate. The obvious thing is to upgrade in `bls12_381` the version of `digest` to v0.10. That would make it possible to get `hkdf` v0.12 built. But such an upgrade is and open issue at zkcrypto/bls12_381#102, which mentions that it's blocked on zkcrypto/bls12_381#90. That pull request is about updating do the hash-to-curve draft v16, currently it's using v12. We use that code path in `bls-signatures`, else we wouldn't enable the `experimental` feature of `bls12_381`. So it's even not clear if we'd want such a change to v16.

* Update `filecoin-proofs-api` to v18 Update `filecoin-proofs-api` to v18 * Bump to 3.10.0 Bump to 3.10.0 * Update cargo.lock and changelog Update cargo.lock and changelog * fix: remove the pairing feature from fvm_shared (#2009) The `pairing` feature from the `fvm_shared` crate isn't used. It causes problems, as it forces the `subtle` dependency to v2.4.1, although the rest is happy to have v2.5.0. Here is a detailed dependency graph and issue outline: `fvm_shared` depends on `bls-signatures`. In `bls-signatures` we depend on an old version (v0.11) of `hkdf`. That version depends on `hmac` v0.11, which depends on `crypto-mac` v0.11. `crypto-mac` v0.11.0 depends on `subtle` v2. That is fine, it would automatically select v2.5.0. The problem is that `crypto-mac` v0.11.1 pins `subtle` to exactly v2.4, therefore v2.5.0 won't be selected. The obvious thing is to upgrade in`bls-signatures` the version of `hkdf` to the latest v0.12. That would make it possible to use `subtle` v2.5.0. The problem is that such an upgrade is not easily possible. `hkdf` v0.12 depends on a newer version v0.10 of the `sha2` crate. Updating that breaks the `bls12_381` crate. The reason is the current version v0.8.0 of `bls12_381` depends on an old version v0.9 of the `digest` crate. The obvious thing is to upgrade in `bls12_381` the version of `digest` to v0.10. That would make it possible to get `hkdf` v0.12 built. But such an upgrade is and open issue at zkcrypto/bls12_381#102, which mentions that it's blocked on zkcrypto/bls12_381#90. That pull request is about updating do the hash-to-curve draft v16, currently it's using v12. We use that code path in `bls-signatures`, else we wouldn't enable the `experimental` feature of `bls12_381`. So it's even not clear if we'd want such a change to v16. * Update Changelog.md with #2009 backport Update Changelog.md with #2009 backport * Update fvm_shared to 3.10.0 Update fvm_shared to 3.10.0 in `testing/integration/Cargo.toml` * Update fvm_shared to v3.10.0 Update fvm_shared to v3.10.0 * Update cargo.lock Update cargo.lock by running `cargo check --all` * Update `shared/CHANGELOG.md` Update `shared/CHANGELOG.md` --------- Co-authored-by: Volker Mische <volker.mische@gmail.com>

* Update filecoin-proofs-api to v18 Update filecoin-proofs-api to v18 * Bump to 2.8.0 Bump to 2.8.0 * Update cargo.lock and changelog Update cargo.lock and changelog * fix: remove the pairing feature from fvm_shared (#2009) The `pairing` feature from the `fvm_shared` crate isn't used. It causes problems, as it forces the `subtle` dependency to v2.4.1, although the rest is happy to have v2.5.0. Here is a detailed dependency graph and issue outline: `fvm_shared` depends on `bls-signatures`. In `bls-signatures` we depend on an old version (v0.11) of `hkdf`. That version depends on `hmac` v0.11, which depends on `crypto-mac` v0.11. `crypto-mac` v0.11.0 depends on `subtle` v2. That is fine, it would automatically select v2.5.0. The problem is that `crypto-mac` v0.11.1 pins `subtle` to exactly v2.4, therefore v2.5.0 won't be selected. The obvious thing is to upgrade in`bls-signatures` the version of `hkdf` to the latest v0.12. That would make it possible to use `subtle` v2.5.0. The problem is that such an upgrade is not easily possible. `hkdf` v0.12 depends on a newer version v0.10 of the `sha2` crate. Updating that breaks the `bls12_381` crate. The reason is the current version v0.8.0 of `bls12_381` depends on an old version v0.9 of the `digest` crate. The obvious thing is to upgrade in `bls12_381` the version of `digest` to v0.10. That would make it possible to get `hkdf` v0.12 built. But such an upgrade is and open issue at zkcrypto/bls12_381#102, which mentions that it's blocked on zkcrypto/bls12_381#90. That pull request is about updating do the hash-to-curve draft v16, currently it's using v12. We use that code path in `bls-signatures`, else we wouldn't enable the `experimental` feature of `bls12_381`. So it's even not clear if we'd want such a change to v16. * Update cargo.lock and Changelog.md Update cargo.locl and Changelog.md * Update fvm_shared, cargo.lock and changelog Update fvm_shared, cargo.lock and changelog --------- Co-authored-by: Volker Mische <volker.mische@gmail.com>

str4d · 2024-07-21T14:52:27Z

The PR has not been split yet by the author, so I am going to take over this PR and do so myself. I'll do the move in a separate PR, and then rebase this PR on top of that.

Extracted from #90.

str4d · 2024-07-21T15:54:45Z

Rebased on main to remove the tests/ move from this PR (and gain access to updated CI).

str4d · 2024-07-21T15:55:28Z

Force-pushed to fix rustfmt issue.

str4d · 2024-07-21T16:11:17Z

Force-pushed to fix benchmarks that were broken by the second commit.

str4d

Reviewed 7c6f4bc, except for changes to the tests.

src/hash_to_curve/expand_msg.rs

tests/hash_to_curve_g1.rs

Signed-off-by: Andrew Whitehead <cywolf@gmail.com>

str4d · 2024-07-21T19:55:44Z

Force-pushed to unswap the test orders, and unwrap test vectors.

str4d

Reviewed db71a01. I have confirmed that the only functional changes in the draft update (that affect this PR) are additional assertions.

Just so others looking at this repo have a sense of time, this PR took me over half of my Sunday to update and review.

…ting The draft (and the final RFC) do not explicitly give an upper limit on the hash output size, but section 5.3.3 says that `expand_message_xmd` requires that DST is at most 255 bytes, and when given a longer domain separation tag, to use the output of `H` directly as the DST. To avoid any ambiguities, we bound the set of `H` we support.

str4d

ACK 09466e0. I'm going to open a follow-up PR that migrates to the now-released RFC 9380 and removes the experimental feature flag.

andrewwhitehead force-pushed the h2c-api-upd branch from 26f5cd7 to 5ca2178 Compare July 5, 2022 21:18

str4d mentioned this pull request Dec 6, 2022

alloc independent hash_to_curve API zcash/pasta_curves#46

Open

daira reviewed Jan 12, 2023

View reviewed changes

tests/hash_to_curve_g1.rs Outdated Show resolved Hide resolved

This was referenced Feb 27, 2023

Update CI #103

Merged

Upgrade sha2/sha3/digest from 0.9 to 0.10 #102

Closed

vmx mentioned this pull request May 24, 2024

fix: remove the pairing feature from fvm_shared filecoin-project/ref-fvm#2009

Merged

str4d added a commit that referenced this pull request Jul 21, 2024

Move hash-to-curve tests to tests/ for compilation performance

5ffba28

Extracted from #90.

str4d mentioned this pull request Jul 21, 2024

Move hash-to-curve tests to tests/ for compilation performance #134

Merged

str4d added a commit that referenced this pull request Jul 21, 2024

Move hash-to-curve tests to tests/ for compilation performance

6d98c36

Extracted from #90.

str4d force-pushed the h2c-api-upd branch from 3102b60 to 495efdc Compare July 21, 2024 15:53

str4d force-pushed the h2c-api-upd branch from 495efdc to 39f36f1 Compare July 21, 2024 15:55

str4d force-pushed the h2c-api-upd branch from 39f36f1 to 7c6f4bc Compare July 21, 2024 16:10

str4d requested changes Jul 21, 2024

View reviewed changes

andrewwhitehead added 2 commits July 21, 2024 19:53

update expand_message API

4b4e7f8

Signed-off-by: Andrew Whitehead <cywolf@gmail.com>

only implement Message for iterators

db71a01

Signed-off-by: Andrew Whitehead <cywolf@gmail.com>

str4d force-pushed the h2c-api-upd branch from 7c6f4bc to db71a01 Compare July 21, 2024 19:55

str4d requested changes Jul 21, 2024

View reviewed changes

str4d added 4 commits July 21, 2024 21:26

Document L generic of ExpandMessage::init_expand

eb3f0a9

Add missing len_in_bytes check to ExpandMsgXmd::init_expand

cf264e6

Remove pub qualifiers from non-pub ExpandMsgDst

9a81247

str4d approved these changes Jul 21, 2024

View reviewed changes

str4d merged commit 2874b5a into zkcrypto:main Jul 21, 2024
12 of 13 checks passed

str4d mentioned this pull request Jul 21, 2024

Upgrade sha2/sha3/digest from 0.9 to 0.10 #130

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to hash-to-curve draft 16, with some API adjustments #90

Update to hash-to-curve draft 16, with some API adjustments #90

andrewwhitehead commented Jul 5, 2022

str4d commented Feb 27, 2023 •

edited

Loading

str4d commented Feb 27, 2023

str4d commented Jul 21, 2024

str4d commented Jul 21, 2024

str4d commented Jul 21, 2024

str4d commented Jul 21, 2024

str4d left a comment

str4d commented Jul 21, 2024

str4d left a comment •

edited

Loading

str4d left a comment •

edited

Loading

Update to hash-to-curve draft 16, with some API adjustments #90

Update to hash-to-curve draft 16, with some API adjustments #90

Conversation

andrewwhitehead commented Jul 5, 2022

str4d commented Feb 27, 2023 • edited Loading

str4d commented Feb 27, 2023

str4d commented Jul 21, 2024

str4d commented Jul 21, 2024

str4d commented Jul 21, 2024

str4d commented Jul 21, 2024

str4d left a comment

Choose a reason for hiding this comment

str4d commented Jul 21, 2024

str4d left a comment • edited Loading

Choose a reason for hiding this comment

str4d left a comment • edited Loading

Choose a reason for hiding this comment

str4d commented Feb 27, 2023 •

edited

Loading

str4d left a comment •

edited

Loading

str4d left a comment •

edited

Loading